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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address » 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )EE3 Responsive to communication(s) filed on 18 March 2004 . 
2a)S This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-16 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) [X] Claim(s) U16 is/are rejected. 

7) D Claim(s) - is/are objected to. - - _ .. . 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 13 The specification is objected to by the Examiner. 

10® The drawing(s) filed on 17 September 2001 is/are: a® accepted or b)D objected to by the Examiner. 
Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)Q None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1 ) □ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) D Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5 ) d Notice of Informal Patent Application (PTO-152) 

Paper No(s)/Mail Date . 6) O Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 
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DETAILED ACTION 



1 . This Office Action is in response to an AMENDMENT entered March 18, 2004 
for the patent application 09/954,755 filed on September 17, 2001 . 

2. The First Office Action of November 13, 2003 is fully incorporated into this Final 
Office Action by reference. 



4. The specification is objected to because of the following: 

The specification at page 1 1 , lines 7-15 cites: "The accuracy 18 provides an 
indication of a degree of correctness in the classification rendered by the classifier 12 
where the correctness is provided by the authoritative classifier 14. The accuracy 18 
may be viewed as providing a measure of distance between the zero or more 
categories selected by the classifier 12 for the item 10 and the one or more categories 
selected by the authoritative classifier 14 for the item 10." The specification does not 
identify a methodology for distance calculation. 

Claim 1 of the specification at lines 7-10 cites "... a set of categories of an 
arrangement of categories selected for an Item by the classifier..." and "... a set of 



Status of Claims 



3. 



Claim 1 is amended. Claims 1-16 are pending. 



Specification 
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categories of the arrangement selected for the item by an authoritative classifier..." The 
writings indicate that both "sets of categories" are distinct or independent 

The specification principally focuses on a measure of accuracy for classifier 12. 
As the calculations are developed for classifier 12 and ultimate in reference to accuracy 
18, the relationship to how the authoritative classifier with its independent set of 
categories effects the accuracy is not addressed at specification pages 7-1 1 . To one of 
ordinary skill in the art, there would be undue experimentation to establish a 
methodology, determine the separate and independent set of categories, and then 
determine the degree of correctness that is based on distance. 

These objections must be corrected. 



Claim Rejections - 35 USC § 112 

5. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

6. Claims 1-16 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. Specifically, the invention focuses on the use of an 
authoritative classifier 14 that is defined by the specification at page 7, lines 15-16, to be 
"...a human expert ..." It is axiomatic that from one human expert to another, 
classifications would be different reducing the claims to a level of uncertainty or 
indefiniteness. 
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7. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

8. Claims 1-16 are rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the enablement requirement. The claim(s) contains subject matter which 
was not described in the specification in such a way as to enable one skilled in the art to 
which it pertains, or with which it is most nearly connected, to make and/or use the 
invention. Para 2 above applies. Specifically, the invention focuses on the use of an 
authoritative classifier 14 that is defined by the specification at page 7, lines 15-16, to be 
"...a highly accurate automated classifier ..." Such a general claims would read on a 
Bayesian classifier using some transformation F (x) on the observed data where such 
data might experience some preprocessing. To apply such a concept without 
disclosing how it would be implemented would require undue experimentation by one of 
ordinary skill in the art. And yet, the claim is of such generality that it covers such a 
situation. Hence, the generality drives the lack of enablement in the specification. 

Claim Rejections - 35 USC § 101 

9. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

10. Claim 1-16 are rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. 
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1 1 . The practical application test requires that a useful, concrete and tangible result 
be accomplished. Claims 1-16 represent abstract methodology with human being 
processing. The abstract methodology establishes rejection on the basis of intangibility. 
Further, it is axiomatic that from one human expert to another, classifications would be 
different establishing rejection on the basis of lack of concreteness. Either rejection 
establishes the consequence of non-statutory. 



Claim Rejections - 35 USC § 102 

12. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

A person shall be entitled to a patent unless - 

13. Claim 1 is rejected under 35 U.S.C. 102(b) as being anticipated by Peele et al 
(USP 5,561,431 referred to as Peele). 

Claim 1 

Peele anticipates determining an accuracy measure which indicates a degree of 
correctness of the classifier in classifying an item in response to a set of categories of 
an arrangement of categories selected by the classifier for the item and a set of 
categories of the arrangement of categories selected by an authoritative classifier for 
the item (Peele, c 8, 1 30-67). 
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Response to Arguments 

14. Applicant's arguments filed on March 18, 2004 related to Claims 1-16 have been 
fully considered but are not persuasive. 
In reference to Applicant's argument: 

The Examiner has objected to the specification. The Examiner has stated that the specification does not 
identify a methodology for a distance calculation. The Examiner has also stated that the specification 
would require one of ordinary skill in the art to engage in undue experimentation to establish a 
methodology, determine separate and independent sets of categories by a classifier and an authoritative 
classifier, and then determine a degree of correctness based on distance. 

37 CFR §1.71 requires that the specification set forth a written description of the invention sought. 
Applicant submits that the invention sought and claimed in amended claim 1 and claims 2-16 does not 
include performing a distance calculation or determining separate and independent sets of categories by 
a classifier and an authoritative classifier or determining a degree of correctness based on distance. 
Therefore, applicant submits that the specification is not required to provide descriptions of performing a 
distance calculation or determining separate and independent sets of categories by a classifier and an 
authoritative classifier or determining a degree of correctness based on distance. Amended claim 1 
recites the step of "determining an accuracy measure which indicates a degree of correctness of the 
-classifier in classifying an item" and Applicant's specification provides ample description of determining an 
accuracy measure. (See Figure 3 of Applicant's specification and accompanying text). 

Examiner's response: 

From the First Office Action at page 3, lines 7-13: The specification principally 
focuses on a measure of accuracy for classifier 12. As the calculations are developed 
for classifier 12 and ultimate in reference to accuracy 18, the relationship to how the 
authoritative classifier with its independent set of categories effects the accuracy is not 
addressed at specification pages 7-1 1 . To one of ordinary skill in the art, there would 
be undue experimentation to establish a methodology, determine the separate and 
independent set of categories, and then determine the degree of correctness that is 
based on distance." The Applicant has not addressed this concern wherein the 
discussion of Fig. 3 begins on page 7 of the specification. The objection to the 



specification remains. 
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In reference to Applicant's argument: 

The Examiner has rejected claims 1-16 under 35 U.S.C. §112, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter applicant regards as the invention. 
The Examiner has stated that the invention "focuses" on the use of an authoritative classifier that may be 
a human expert and that the claims are therefore indefinite because the classifications performed by 
different human experts could differ. 

Applicant respectfully submits that the invention as claimed in amended claim 1 and claims 2-16 does not 
"focus" on the classifications performed by an authoritative classifier. Amended claim 1 and claims 2-16 
do not include any language for performing classifications. Instead, claims 1-16 recite method steps for 
determining an accuracy measure for a classifier in response to a set of categories selected by the 
classifier and a set of categories selected by an authoritative classifier. 

Examiner's response: 

No changes have been made to claims 2-16. Applicant agrees with the 
Examiner with the statement "...claims 1-16 recite method steps for determining an 
accuracy measure for a classifier in response to a set of categories selected by the 
classifier and a set of categories selected by an authoritative classifier." Rejection of 
claims 1-16 under 35 USC 112, second paragraph remains. 



In reference to Applicant's argument: 

The Examiner has rejected claims 1-16 under 35 U.S.C. §112, first paragraph, as failing to comply with 
the enablement requirement. The Examiner has stated that the invention focuses on the use of 
authoritative classifier 14 and that the specification does not disclose how to implement the authoritative 
classifier 14. Applicant respectfully submits that the invention as claimed in amended claim 1 and claims 
2-16 is not directed to an authoritative classifier. Instead, claims 1-16 are directed to determining an 
accuracy measure for a classifier in response to a set of categories selected by the classifier and a set of 
categories selected by an authoritative classifier. Claims 1-16 do not include any language pertaining to 
the operation of an authoritative classifier which would require description in the specification. The 
classification steps performed by the authoritative classifier are outside of the scope of claims 1-16. 

Examiner's response: 

Again, the applicant agree with the Examiner with the statement: "...claims 1-16 
are directed to determining an accuracy measure for a classifier in response to a set of 
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categories selected by the classifier and a set of categories selected by an authoritative 
classifier." Rejection of claims 1-16 under 35 USC 112, first paragraph remains. 



In reference to Applicant's argument: 

The Examiner has rejected claims 1-16 under 35 U.S.C. §101 as being directed to non-statutory subject 
matter. The Examiner has stated that claims 1-16 represent abstract methodology with human being 
processing. The Examiner has stated that from one human expert to another classifications would be 
different and that the claims therefore lack concreteness. Applicant submits that the human being 
processing to which the Examiner refers is classifications performed by a human expert. 

Applicant respectfully submits that amended claim 1 and claims 2-16 do not include classification steps 
performed by human experts. Instead, claims 1-16 are directed to determining an accuracy measure for a 
classifier in response to a set of categories selected by the classifier and a set of categories selected by 
an authoritative classifier. The classification methodology performed by the authoritative classifier and the 
classifier being evaluated are outside of the scope of claims 1-16. Furthermore, applicant's specification 
sets forth numerous practical applications of a method for determining an accuracy measure according to 
claims 1-16, e.g. designing a classifier, training a classifier, evaluating/improving system models, 
comparing the performance of different classifiers, etc. 

Examiner's response: 

Once again, the applicant agrees with the Examiner with the statement: 

"...claims 1-16 are directed to determining an accuracy measure for a classifier in 

response to a set of categories selected by the classifier and a set of categories 

selected by an authoritative classifier." Applicant is directed to review the specification 

at page 7 lines 15-16 where: The authoritative classifier 14 may be a human expert ..." 

Further, the specification at page 7 lines 18-20, states: "The accuracy evaluator 16 may 

be a human being or a group of human beings or an automated system that performs 

the present methods." Applicant further admits to the human presence by stating that: 

"... the human being processing to which the Examiner refers is classifications 

performed by a human expert." The rejection of claims 1-16 under 35 USC 101 



remains. 



Application/Control Number: 09/954,755 



Art Unit: 2121 



Page 9 



In reference to Applicant's argument: 

The Examiner has rejected claim 1 under 35 U.S.C. §102 (b) as being unpatentable over Peele. 
Amended claims 1 is a method for determining accuracy of a classifier that includes determining an 
accuracy measure which indicates a degree of correctness in classifying an item in response to a set of 
categories selected by the classifier and a set of categories selected by an authoritative classifier. Peele 
does not disclose determining an accuracy measure in response categories selected by an authoritative 
classifier as claimed in amended claim 1 . Instead, Peele discloses a classifier that determines a 
confidence measure in response to the classification computations generated by the classifier for selected 
and unselected classes. (Peel e, col. 8, lines 53-60). 

For example, Peele teaches that 

A suitable level of confidence may be generated for each classifier by determining the difference 
CL ch osen - CLother, where CL ch osen refers to the classifier's computed value for the selected class, 
and CL 0 th e r is the computed value for the non-selected class. (Peele, col. 8, lines 56-60). 

In contrast to computing a confidence level in response to classifier values for selected and non-selected 
classes as taught by Peele, an accuracy measure according to amended claim 1 is based on a set of 
categories selected by the classifier being evaluated and a set of categories selected by an authoritative 
classifier. 

Examiner's response: 

Para 15 below applies. Peele's confidence measure is synonymous with the 
applicant's accuracy measure. For sure, Peele does not use a human element in his 
confidence determination. The rejection of claims 1-16 under 35 USC 102(b) remains. 



Examination Considerations 

1 5. The claims and only the claims form the metes and bounds of the invention. 
"Office personnel are to give the claims their broadest reasonable interpretation in light 
of the supporting disclosure. In re Morris, 127 F.3d 1048, 1054-55, 44USPQ2d 1023, 
1027-28 (Fed. Cir. 1997). Limitations appearing in the specification but not recited in 
the claim are not read into the claim. In re Prater, 415 F.2d, 1393, 1404-05, 162 USPQ 
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541, 550-551 (CCPA 1969)" (MPEP p 2100-8, c 2, 1 45-48; p 2100-9, c 1, 1 1-4). The 
Examiner has full latitude to interpret each claim in the broadest reasonable sense. 
Examiner will reference prior art using terminology familiar to one of ordinary skill in the 
art. Such an approach is broad in concept and can be either explicit or implicit in 
meaning. 

16. Examiner's Notes are provided to assist the applicant to better understand the 
nature of the prior art, application of such prior art and, as appropriate, to further 
indicate other prior art that maybe applied in other office actions. Such comments are 
entirely consistent with the intent and spirit of compact prosecution. However, and 
unless otherwise stated, the Examiner's Notes are not prior art but a link to prior art that 
one of ordinary skill in the art would find inherently appropriate. 

17. Examiner's Opinion 

Para 15 above applies. Among other concerns, this application has a serious 35 
USC 101 problem wherein the authoritative classifier among other similar items is very 
specifically human linked. 



Conclusion 

18. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
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§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .1 36(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 
19. Claims 1-16 are rejected. 



Correspondence Information 

20. Any inquiry concerning this information or related to the subject disclosure 
should be directed to the Examiner, Joseph P. Hirl, whose telephone number is 
(703) 305-1668. The Examiner can be reached on Monday - Thursday from 
6:00 a.m. to 4:30 p.m. 

If attempts to reach the Examiner by telephone are unsuccessful, the 
Examiner's supervisor, Anthony Knight can be reached at (703) 308-3179. 
Any response to this office action should be mailed to: 
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Commissioner of Patents and Trademarks, 
Washington, D. C. 20231; 
or faxed to: 

(703) 746-7239 (for formal communications intended for entry); 
or faxed to: 

(703) 746-7290 (for informal or draft communications with notation of 
"Proposed" or "Draft" for the desk of the Examiner). 
Hand-delivered responses should be brought to: 

Receptionist, Crystal Park II 

2121 Crystal Drive, 

Arlington, Virginia. 





Joseph P. Hirl 



May 24, 2004 



